Evaluating Instructional Hypermedia: A User Modelling Perspective

The goal of this research is to define a semi-automated method called Applying Patterns to Hypermedia Instructional Design (APHID) for the development of instructional hypermedia applications using established software engineering and instructional design principles. Resulting applications are evaluated using techniques common to user modelling and adaptive hypermedia. 1 Applying Patterns to Hypermedia Instructional Design (APHID) The APHID method is based on the notion that design patterns can be used to describe instructional design principles and subsequently used to guide the design of instructional hypermedia applications. The research spans several fields including software engineering, instructional design, hypermedia design, and user modelling. Of interest to the user modelling conference is the novel application of user modelling techniques to the validation of the hypermedia applications produced using the prototype APHID environment. One important aspect of the instructional process is sequencing information for presentation to learners (Merrill, 1998). Within a hypermedia application, instructional design is accomplished, in part, by carefully planning the learner’s sequence through the material. Most hypermedia development processes provide no mechanism for including the sequencing of conceptual information in the application design. The APHID method documents the structure associated with the concepts in the domain of instruction as a concept map. Instructional design is represented as organizational patterns on the concept map. Both are used during the creation phase to automatically generate instructional hypermedia applications that are tailored to a specific instructional goal. Because the creation process is not completely controlled by the instructional designer, it is paramount that validity and usability of the hypermedia applications generated by APHID be established. A significant portion of this research is to evaluate the applications produced using the APHID environment. 2 Evaluating Hypermedia Most effective software validation exercises include some evaluation of the software by users (Nielsen & Mack, 1994). Validation of user interactions with hypermedia is limited by what can be observed either directly or through some sort of monitoring. One observable characteristic of hypermedia users is the path they take through an application. A record of user browsing behaviour through a hypermedia application allows comparison of different users and classification of those users into groups. A user who is classified into one particular group is likely to benefit from pages viewed by other users who also belong to that group. Much of the present work on user interactions with hypermedia comes from user modelling researchers who wish to create adaptive hypermedia. One method of classifying users relies on a longest common subsequence algorithm for determining the similarity of one path through the hypermedia to another path (Sun & Ching, 1995). Different users of the same hypermedia application should have similar paths through the hypermedia, if their purposes and skill levels are relatively equal. A second method considers not the path through the hypermedia, but the material viewed by the user (Yan, Jacobsen, Garcia-Molina & Dayal, 1996). Each page the user accesses, along with the approximate time spent on the page is represented as part of a vector. The vectors from different users are clustered and the clusters are used as classifications for the users. When these techniques are used to adapt hypermedia applications for individuals the hypermedia is assumed to be correct. Once the individual user’s preferences (or purpose) are identified, a correct adaptation can be supplied. A slightly different perspective allows the same algorithms to be used to evaluate the hypermedia application. If the purpose of a group of users is assumed, their browsing patterns should be similar to one another if the application is constructed properly. In the case of an APHIDcreated application, the purpose is assumed, therefore users with similar backgrounds and abilities should browse through the hypermedia application using a similar pattern. The application of previously mentioned techniques to browsing logs should result in most users falling in the same cluster if the application is correctly constructed. For this research, a controlled experiment using browsing pattern analysis will be performed to determine if APHID-created hypermedia applications are valid. The users will be selected to be homogeneous (all from the same beginning computer science class). These users should have similar browsing patterns (both in terms of pages viewed and in terms of order) and those patterns should match the desired patterns predicted by the APHID environment. It is hoped that users who self-report an increase in knowledge after using the hypermedia application will be the same users who exhibit browsing patterns that match the patterns predicted by APHID. Second, we hope to show that users working with a properly constructed instructional application exhibit similar (and predictable) browsing patterns. Future applications created with APHID can then be validated simply by analyzing user browsing patterns.